Scheduled Imports

Overview

In this document, the concepts and processes of setting up Scheduled Imports will be discussed. Setting up and editing Scheduled Imports can be done by your own team or by Savance Support.

Assumptions

We will assume throughout this article that you use external data that you want to import into Savance and want to automate and schedule the imports.

General Concepts

• Access the Scheduled Imports screen by clicking on Admin, Import, then Scheduled Imports.

• Any Scheduled Imports will be listed in this screen and can be edited, deleted, manually processed (process now button), or refreshed as needed.

• To add a new scheduled import, click the New button.

• Start by Naming the scheduled import.

New Profile Source Type

• For Import source profile, you can either choose one or add a new profile by clicking Import Profile.

• Name the profile then choose what source type you want to import. FTP, Share, or URL.

• Depending on the source type, fill out the settings section accordingly.

FTP

• An FTP site can be used to transfer files, backups, videos, media files, etc.. Not only web site data. The FTP protocol is one of the most popular nowadays. FTP is used to transfer files from a website and is optimized for a large number of files, and bigger file sizes as well.

• Enter the site and the Port.

• Enter username and password for the site.

• Paste the folder location.

• Choose what security method to use: SSL, TLS, passive.

• Decide when to delete files.

Share

• Use the “...” button to enter the file name, or select a whole folder.

URL

• If the files you need to import are on a webpage, you can use URL.

  • Enter the folder/file name, and the URL/website you’re pulling from.

  • If you use HTTP, choose the method (get, post, or put) and enter the parameters.

  • If your file/folder name and/or URL name are dynamic, click on the “...” button by those fields to set up the naming convention for them.

    • Drag and drop the variables into the naming space to build the name.

• Next, fill out the bottom half of the new profile screen, setting up the Filter Settings.

  • Indicate what file extension it is (such as CSV)

  • OR choose a RegEx type

• The rest indicates what files in the folder you want it to look at and if you want only the top 1 or even all files, and select a sort order if not all files in the folder.

• The section to the right is if the files are in a zip folder, this essentially tells it to unzip the folder (fill out both left and right sections if it’s a zip folder and the file extension on the right would be “zip”).

• When you’ve finished creating the new import profile, click the Save/close button.

• Back in the new scheduled import screen, select the Import Template.

New Import Template

• You can also create a new import template.

• Click on the Import Template and select Add New Template (or edit an existing one if needed).

• In the Template screen, choose the source file and set up your mapping.

• When ready, click OK.

• Back on the Schedule Import screen, choose a schedule. If you want this import to be scheduled at the same time as others, you can just add it to that schedule. Or create a new one.

  • You can also edit a schedule by selecting one and then clicking Schedule Settings then Edit Schedule.

Add New Schedule

• To add a new schedule, click Schedule Settings and click Add New Schedule.

• Name the schedule and indicate if it’s to be ran once, on startup, or recurring.

  • Depending on which schedule type you choose, the different sections of the screen will be enabled.

• When ready, click Save and close to get back to the scheduled imports set up screen.

Automated Commits and Conflict Resolution

• Back on that screen, you will then select the automated Commits and Conflict Resolution setting.

  • This is what validates the data it imports.

• “No automated Commits” the process would just run the schedule and leave the file/s in the queue with “action required” status in the Import History screen. Then the user would need to go into the Import History screen and manually commit the import.

  • Remember, all imports that you did not set up to be automatically committed must be manually committed on this screen.

• “Always Automate Commit” would always commit the import, but will bypass any errors found.

• “Automate Commit if No Errors” will leave the imported file/s in the queue with “action required” status only if it found errors with the import.

  • The checkbox below this option essentially sees new rows as an error and does not commit if there are new rows.

  • If it finds errors, the user would need to fix any errors and retry the run (or wait till the next scheduled run, depending on the nature of the file).

Contacts for File Imports

• Lastly is to set up who will be contacted when the file transfer happens. Emails will be sent out when the import is successful and when there is an error.

Tip for File Import Notifications

If you have a schedule that runs many times a day/week and you’re set up as a contact, you can set up a rule on your email provider that sends the emails to a different folder. For example, if you only want to see the email for when the import errors, you can have the successful emails be sent to a folder and marked as read so you don’t have to see them flood your inbox, and you only see the error notifications in your inbox. This is just one of many ways you can manage and keep your email notifications clean with regard to Scheduled Imports, since it does send an email for every run.

• When ready, click OK and go back to the scheduled imports setup screen.

• Click Save, if you want to run the import now, you can do so by clicking Process Now. Otherwise, you can click Save/Close to finish.

Import History and Errors

• You can see the runs made by clicking Admin, Imports, and Import History. This is where you can find the errored runs or commit imports that are not yet committed.

• If the status shows Error Processing there was an error during processing and could not finish.

• If the status is Finished Processing but has Action Required, it either has not auto committed or it finished uploading but with an error.

• This is where the “Automated Commits and Conflict Resolution” part of the set up comes into play. So there could be some that always have “action required” or some that only need action when it’s errored. The email notification should say whether it was an error or if it is just not set to autocommit.

Error Processing

• If you click on the “error processing” status link, the “Exception Info” window will appear explaining what went wrong. The fix for this is typically just re-run the import or just wait till the next scheduled run.

Action Required

• Click on the Action Required status link to pull up the Import Results to view the data it pulled. If there were no errors, you can just click Commit at the bottom right side of the screen.

• An error could be new lines, strange characters, or unexpected formatting in a cell. You can find and make the changes in the individual cell where needed. Then you can either use the Ignore Changes button or Recheck it.

• Then, if you are ready, you can click Commit. A warning at the top of the screen says that errors found will be skipped, so if there are more errors, they will simply be skipped in the commit.

• Remember, all imports that you did not set up to be automatically committed must be manually committed on this screen.